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ABSTRACT 


Supervisory control, where control Is traded between man and computer, 
may offer benefits In the control of a remote manipulator. A system for the 
study of supervisory control Is described, and some preliminary results pre- 
sented . 


I. INTRODUCTION 


Man's technological rise has been accompanied by an Increasing need for 
Mm to work In hostile environments. Nuclear waste disposal sites, radio- 
active laboratories, the depths of the ocean, the vacuum of outer space, and 
underground mines are examples of such environments. Teleoperator systems 
project man's manipulatory capabilities Into the remote environment, allow- 
ing his functional presence without his physical presence. 

The need for teleoperators Implies operating conditions which exclude 
or impair visual or other human sensory contact between the operator and the 
manipulator. The barrier Imposed by the hazardous environment carries with 
It limits on both sensor and control communication. 

It Is possible ,for a man and computer to cooperate In the control of a 
remote manipulator, and achieve a standard of performance beyond that possi- 
ble by either alone. This mode of control has been termed supervisory con- 
trol [1]. 

The first manipulators were master-slaves used for radioactive materi- 
als handling. They provided bilateral force feedback, and the operator 
viewed the site directly a few feet away. In this environment the operator 
was capable of precise positioning and fine control of applied forces. How- 
ever, 4s the linkage between human operator and manipulator was made elec- 
trical, and the distance Increased, the Impairment of sensory feedback made 
control more difficult. 

The first sensory degradation that was Investigated was a transmission 
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time-delay between the operator's command and the manipulator's response. A 
time-delay can arise from extreme separation (outer space manipulation), or 
sensor/di splay limitations such as limited video frame rate, 

"err '11 [2], Black [3], Hill and Sword [4], and Slarr [5], experimen- 
tally verified the negative effects of time-delay on manipulation. Simple 
tasks became frustrating and laborious, and manipulation requiring high 
accuracy was virtually impossible. 

Supervisory control may !illow manipuluion to be performed effectively 
even when a time-delay 1;, present. The philosophy of supervisory control is 
that the human operator plans strategy, monitors performance, and intervenes 
when necessary, while the computer accomplishes portions of the task as 
instructed by the htxnan. Supervisory control allocates control responsi- 
bility between man and computer such that the inherent attributes of each 
are used to best advantage. 

Figure 1 shows diagrams of manual control and supervisory control of a 
remote manipulator system. Under manual control the operator's commands are 
sent directly to the manipulator, and feedback from its sensors is displayed 
directly to him. Under supervisory control the operator's commands are 
transmitted to a remote computer, which then commands the manipulator. The 
remote computer also processes the sensory information from the manipulator 
and relays it to the operator. Since the remote computer is "on site" it is 
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not subject to the same feedback degradation as the human operator. 
portion of the system comprised by the manipulator and remote computer can 
accomplish portions of the task on its own. The human functions principally 
as coordinator and supervisor, but may still control the manipulator 
directly if necessary. 


II. SYSTEM OVERVIEW 


Figure 2 shows a block diagram of the system. There are three seoarate 
digital computers, each with its own responsibilities. The Manual Cortrol 
Computer (MCC) processes manual control signals originating from the human 
operator. The Arm Control Computer (ACC) performs arm trajectory calcula- 
tion, monitors the jaw-mounted tactile sensors, and drives the arm through 



BLOCK DIAGRAM OF SUPERVISORY 
CONTROL SYSTEM 

Figure 2 


desired positions. The Supervisory Control Computer (SCC) monitors the 
sensors, sends commands to the ACC, and performs quantitative analysis of 
arm positions and sensor data. 

The SCC and ACC are coupled by a 9600 band serial communications link. 
The ACC executes the VAL language while the SCC uses the RT-11 operating 
system, with programs written in FORTRAN or assembler language. 
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Example of Systjm Operation. Although the Individual elements in the 
system will be described later, the following alignment routine (used in the 
experiments) is an example of how they work together. 

Consider the operation of removing a nut from a stud, when the plane on 
which the stud is fastened is arbitrarily oriented and the manipulator hand 
is "far away" from the nut. Before a pre-programmed nut-removal routine can 
be executed, the nut must be approached, and the hand must be "lined up" 
with the axis of the nut, i.e., normal to the plate. The sequence of oper- 
ations is as follows (see Figure 3): 

1. The human operator brings the hand to the vicinity of the nut, man- 
ually orients the hand to be roughly normal to the plate, then instructs the 
see to execute the "ALIGN" routine. The SCC commands the ACC to execute its 
stored alignment routine. The ACC advances the hand in the direction it is 
pointing until it touches the plate at location 1. 



1. MAN MOVES ARM TO TOUCH AT ”l" 

2. COMPUTER MOVES ARM TO TOUCH AT *2* AND" 3* 

3. VECTOR fi • ^ x COMPUTED 

4 COMPUTER ORIENTS HAND ALONG C 
& CONTROL RETURNED TO MAN 

ALIGNING HAND WITH BOLT AXIS 

Figure 3 
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2 . The ACC detects the touch, stores the position where the touch 
occurred, and moves the hand to touch the plate at locations 2 and 3, stor- 
ing these locations. 

3. The three locations define vectors A and from which vector ^ is 
computed by the ACC. 

4. The ACC drives the hand to point along £. The hand is now aligned 
with the nut axis. 

5. The see notifies the human operator that the hand is aligned, and 
displays the options for continuation, which may include resumption of man- 
ual control, or invocation of another computer routine. 

The remainder of the paper will be devoted to describing the elements of the 
system and discussing a preliminary experiment and results. 


III. MANIPULATOR 


The manipulator is a PUMA 600 manufactured by Unic.iation, Inc. The arm, 
shown in Figure 4, has six revolute joints, each powered by a DC servomotor. 
The arm has a position repeatibility of 0.1 mm (0.004 in.). It can apply a 
static force of 58.0 N (13.0 lb). Its length is approximately one meter. 

Each arm joint is under the local control of an R6503 8-bit micropro- 
cessor, which receives commands from the Arm Control Computer (ACC), a DEC 
LSI-11. Each 6503 has its own PROM, RAM, and I/O logic, and servos its 
joint via a D/A converter and shaft encoder. 



Figure 4 
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IV. MANUAL CONTROL SYSTEM 


Resolved motion rate control (RMRC) is the manual control mode. With 
RMRC, the operator uses a six degree-of -freedom isometric joystick to speci- 
fy velocity components of the hand along axes of a hand -mounted cartesian 
coordinate system. The hand-mounted coordinate system is shown in Figure 5. 

The six degree-of-freedom isometric joystick, shown in Figure 6, pro- 
duces six output signals proportional to the six applied forces and torques. 
The magnitude of a force/torque coiiiponent determines the value of corres- 
ponding translational/rotational velocity component. Thus, if the operator 
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Figure 5 


pushes forward on the joystick, the hand moves forward (in the direction it 
is pointing); if he pushes down, it moves down, etc. The coordinate trans- 
formations necessary to drive the hand along the cartesian directions are 
performed by the Arm Control Computer (ACC), which computes arm positions 
each 28 msec. 

The six velocity commands from the joystick do not go directly to the 
ACC, but are first processed by the Motorola 6800-based Manual tontrol 
Computer (MCC). The MCC does the following: 


- 100 - 


ORKstNAL PAGE IS 
OF POOR C JALITY 



6 DOF JOYSTICK 
Figure 6 


1 . Provides nonlinear joystick force/hand velocity relationship . To 
get good nanua! control at low hand velocities, and still be able to command 
high speed without excessive joystick force, a cubic force/velocity rela- 
tionship is implemented. Done in real time with an AMD 9511 floating point 
processor at a s<imp’. log rate of 50 per second, the cubic relation is shown 
in Figure 7. We obtain low sensitivity for low joystick force, and high 
sensitivity for high force. Thus, the arm can be easily driven at its mini- 
mum speed, and may still be driven at maximum speed without requiring exces- 
sively large or small joystick force. 

2. Implement a time-delay . To realistically evaluate the supervisory 
control manipulation system, a communication barrier must be imposed between 
human operator and arm. The most studied, and easiest to systematically 
vary, is a pure time-delay. A time-delay between joystick and arm is pro- 
duced by the MCC, and may be varied from zero to 10 sec, with 0.02 sec reso- 
lution. 

3. Joystick deadban ds. Since velocity control is an integrating pro- 
cess, any hysteretic joystick signal after the operator releases the joy- 
stick will cause undesired arm motion. Software deadbands in the MCC pre- 
vent this. 

4. Computer control takeover . When the operator wants to switch to 
computer control mode, he depresses a button. On receipt of this signal, 
the *CC immediately switches the ACC to computer mode (from manual mode), 


- 101 - 



HAND VELOCITY VERSUS JOYSTICK FORCE 
Figure 7 


then after 0.05 sec the NCC issues a command to the SCf. This causes the 
see to either prompt the operator for a cornnand, or hp^in a preprogrammed 
activity in conjunction with the ACC. The 0.05 sec wait is necessary to 
allow for mode> switching time in the ACC. 

Note that even though this control mode is denoted manual control, it 
is actually a form of computer assistance where the MCC and ACC are in 
series with the human operator. 


V. SENSOR 


A Jaw*mounted tactile sensor is used as the arm-based environmental 
sensing element. The sensor consists of a number of arrays of rectangular 
plates, each of which is sensitive to force. The sensor /jaw appears in Fig 
ure 8. The sensor arrays are connected to the SCC and the ACC via a multi- 
plexer, so that when a contact is sensed, both the face of the Jaw and the 
particular olate responsible for the contact are known. 
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TACTILE SENSOR JAWS 
Figure 8 


VI. COMPUTER CONTROL SYSTEM 


The computer cun!;, .i system consists of two separate computers, the ACC 
(Arm Control Computer) and the SCC (Supervisory Control Computer). Both are 
DEC LSI -11 microcomputers. The ACC and the SCC work together in controlling 
the manipulator. Some functions performed by each machine are: 

Supervisory Control Computer (SCC) : 

1. Can cause execution of a given ACC arm control program. 

2. Read and store arm positions, numerically analyze them, take 
action. 

3. Scan sensor, take appropriate action on contact. 

4. Display prompting or status information to human operator. 

Arm Control Computer (ACC) : 

1. Calculate trajectories, drive arm at desired speed. 

2. Modify stored arm positions to be relative to new reference. 

3. Read sensor, use data to govern program branching, cause jump to 
subroutine. 

4. Execute stored routine upon request from SCC. 

While some functions are common to both, the SCC controls the arm at 
a higher level, selecting ACC programs to executed, analyzing positions. 
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communicating with the operator. The ACC does the lower-level arm control, 
calculating trajectories and issuing joint commands. 

One extremely useful capability uf the ACC is that of relative posi- 
tions. Preprogrammed routines can be "taught" in the laboratory, w'th all 
positions being defined relative to some reference position ("position" 
implies here a six-dimensional quantity including position and orientation), 
which is also defined at this time. When the preprogrammed routine is 
invoked in the field, the reference location will in general be different 
than that in the laboratory. The first instruction in the ACC routine can 
be a redefinition of the reference position to that which exists at that 
time . Upon execution of the routine, the ACC transforms all relative po^si- 
tions to be relative to the newly defined reference position. This yield 
the same relative hand motion regardless of the hand orientation at the time 
the preprogrammed routine is executed. 

The ACC executes the VAL language, an interpretive language which 
resides in 16K EPROM. At the user level, VAL is based on common English 
language words, such as "move," "draw," etc., simplifying program develop- 
ment. 


The see uses the RT-11 operating system, and executes programs written 
in FORTRAN or assembler language. Communications between the ACC and SCC 
scanning of the tactile sensor, and other utility functions are done by 
assembler language subprograms. Quantitative processing of arm position and 
sensor information can be done in FORTRAN. It should be noted that the SCC 
and ACC work asynchronously, and while the SCC is processing data the ACC is 
still controlling the arm. 

To concli'de this section, the nut removal task discussed in Section II 
will be pursued further. In Section II, the hand was aligned with the bolt 
axis (Figure 3). The second phase of the task is to actually determine the 
location of the axis, ihis may be done using the tactile sensor as follows 
(see Figure 9): 

1. After alignment, the human can instruct the SCC to execute the 
"LOCATE" routine. The SCC then prompts the operator to switch to computer 
mode. 


2. The ACC moves the hand until one face of the jaw contacts the nut, 
stores this position, withdraws the jaw, then makes a contact on the oppo- 
site side. This defines line A. 

3. The hand rotates 90 degrees, and two more contacts are made, thus 
defining line B. The SCC determines the intersection of A and B, thereby 
finding the axis. 

If a time-delay is present, this task will likely be much more quickly 
done using supervisory control than manual control. In addition, a lower 
workload will be imposed on the human operate.. The preliminary experiment 
confirmed this. 
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A prel1«inary nanlpMlatlon axp^rinant ms parforaed, priairily to ttai 
the viorking of all of tht systai coaqNifiMts. Tho axpittrlflent eonsfstiit of 
the ranoval of a nut fhM a bolt, m6 usod both tho "ALIGN" «id "LOCATE* 
routines described ew'ller. In addition, a nut roMival routine was devel- 
oped to unscrew the nut ffoi the bolt. 

The operator viewed the task using a fixed, black-and-white TV can«ra 
and mnltor, and was physically Isolated frm the mmlpulator by a parti- 
tion. The task began with the Manipulator hand posltloniid Just at the edge 
of the camera field of vision, and randomly mlsail^ied (pitch of p1us/m1nu$ 
20 degrees, yaw of pius/minus 20 degrees) relative to tke table on which the 
nut was tightened. The task ended when the operator harJ successfully un- 
screwed the nut from the bolt and raised It away fnmi the table surface a 
small amount. 

The task was performed both with and without computer assistance, and 
at two time-delays, 0.0 and 1.0 seconds. Iho replications at each of the 
four conditions yielded a total of el^t trials. Only one subject partici- 
pated In the experiment. 
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Under n.anual control, the subject used the six-axis joystick to control 
the manipulator throughout the experiment. Using computer assistance, the 
subject brought the hand near the table, then typed "ALIGN" to the SCC. 

After completion of the alighment routine, the subject brought the hand 
roughly alongside the nut and typed "LOCATE" to the SCC. After completion 
of this routine the hand was centered over the nut and ready for grasping. 
The subject then typed "NUT" and the manipulator unscrewed the nut under 
computer control. The LOCATE and NUT routines could be combined, but we 
preferred to leave LOCATE more general for future uses other than nut 
removal . 

The completion times for this task are shown in Table 1. Although the 
sample sizes are much too small for statistical comparison, it is evident 
that supervisory control yielded lower completion times, especially at the 
! .0 second time-delay. In addition, the subject reported a definite prefer- 
ence for the computer assistance. 


VIII. CONCLUSION 


This paper has described a system for the study of supervisory control 
of remote manipulation where control is traded between man and computer. By 
virtue of the excellent positional accuracy of the manipulator and the 
effective distribted processing architecture made possible by the VAL soft- 
ware of the Arm Control Computer and the FORTRAN/ Assembler language of the 
Supervisory Control Computer, this system has the potential to accomplish 
sophisticated tasks. The human operator becomes a scene analyzer, planner, 
and supervise- Preliminary experiments indicate that supervisory control 
with this system yield lower task completion times and is preferred by the 
operator over manual control. 


Table 1 
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SUPERVISORY CONTROL 
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163 

113 


166 

106 

1.0 

313 

m 


280 

125 


COHPLETION TiMES^ 

Preliminary Experiment 
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